An analysis on neural dynamics with saturated sigmoidal functions

Ridge Functions, Sigmoidal Functions and Neural Networks

This paper considers mainly approximation by ridge functions. Fix a point a 2 IR n and a function g : IR ! IR. Then the function f : IR n ! IR deened by f (x) = g(ax), x 2 IR n , is a ridge or plane wave function. A sigmoidal function is a particular example of the function g which closely resembles 1 at 1 and 0 at ?1. This paper discusses approximation problems involving general ridge function...

Lower Bounds on the Complexity of Approximating Continuous Functions by Sigmoidal Neural Networks

We calculate lower bounds on the size of sigmoidal neural networks that approximate continuous functions. In particular, we show that for the approximation of polynomials the network size has to grow as O((logk)1/4) where k is the degree of the polynomials. This bound is valid for any input dimension, i.e. independently of the number of variables. The result is obtained by introducing a new met...

Feedforward neural networks with sigmoidal and radial basis functions hidden neurons

This discussion paper concentrates on a possible combination of different activation functions in the multilayered perceptron (MLP) for classification tasks. The activation functions, which are combined are the sigmoidal and the radial basis function. We propose this approach to modification of MLP as solutions to some of the shortfalls of this network.

Multi-sigmoidal units and neural networks

Finiteness Results for Sigmoidal "Neural" Networks

1 1+x 2. This shows that arbitrary (not exp-ra denable) analytic functions may result in architectures with in-nite VC dimension. (Moreover, the architecture used is the simplest one that appears in neural nets practice.) Note that if we wish the x i 's to be bounded, for instance to be restricted to the interval [01; 1], one may replace the above x i 's and w j 's by xi c and cw j , where c = ...